feat: let the widget host pick voice, text, or both - #226
Open
renal128 wants to merge 36 commits into
Open
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
Keep lifecycle state focused while exposing successful startup metrics to callers. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Use ConversationStartResult throughout startup so the connection layer and public session API share one result model. Co-authored-by: Cursor <cursoragent@cursor.com>
Make ConversationStartResult non-discardable by default so callers must consume it or explicitly opt out. Co-authored-by: Cursor <cursoragent@cursor.com>
Ignore late startup-stage callbacks after a conversation has ended or failed so they cannot restore a connecting state. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Wait for server metadata before reporting startup success, using a one-shot per-connection waiter and returning the metadata-derived call info. Co-authored-by: Cursor <cursoragent@cursor.com>
Own fresh single-use Conversation sessions behind a reusable public client so state, callbacks, and lifecycle cleanup remain coherent across starts. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Bind the replacement session before ending its predecessor so overlapping starts cannot reclaim client ownership. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the desired mute state on ConversationClient so it applies before startup and across replacement sessions. Co-authored-by: Cursor <cursoragent@cursor.com>
Stop disconnecting while waiting for the agent from surfacing as agentTimeout via shared onError; keep true timeouts as failures. Co-authored-by: Cursor <cursoragent@cursor.com>
Move tool, end-call, and audio-alignment coverage into Conversation and event-handler tests so suite names match what they exercise. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep incoming-event dispatch next to the session lifecycle so the behavior is easier to find than in a separate extension file. Co-authored-by: Cursor <cursoragent@cursor.com>
Conversation is module-internal and owned by ConversationClient; keep the file layout aligned with that boundary. Co-authored-by: Cursor <cursoragent@cursor.com>
Expose durable mic and agent audio observers with optional level analysis while keeping real-time callbacks off the main actor. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
ConversationAudioObserver replaces track-based advanced audio access, so drop ElevenLabs LocalAudioTrack/RemoteAudioTrack/AudioTrack aliases and point usage docs at the observer API. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Let ClientToolResultEvent own string and Encodable result creation so the client and session expose one focused send method. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Feedback is available whenever the conversation is connected, so consumers can derive availability directly from state. Co-authored-by: Cursor <cursoragent@cursor.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
renal128
force-pushed
the
widget-conversation-modes
branch
from
July 26, 2026 13:54
9a3c8b5 to
bb90972
Compare
renal128
force-pushed
the
widget-conversation-modes
branch
from
July 26, 2026 13:59
bb90972 to
aea208f
Compare
renal128
force-pushed
the
widget-conversation-modes
branch
from
July 26, 2026 16:54
e3aaa49 to
3388337
Compare
renal128
force-pushed
the
widget-conversation-modes
branch
from
July 26, 2026 16:59
3388337 to
a4ace7f
Compare
Add Endpoints on ConversationConfig and wire them through TokenService and the connection managers. HTTP requests derive from a single apiBase; ws/wss endpoints stay explicit. Drop ConnectionConstants and the global apiEndpoint/websocketUrl overrides. Co-authored-by: Cursor <cursoragent@cursor.com>
Promote LogLevel, AgentState, and version to module-level API, move Events under Conversation, and replace ElevenLabs.configure with ConversationClient(logLevel:). Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Use a single EndReason for ConversationState.ended and onDisconnect, and give endConversation one reason parameter. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the one-field wrapper with agentReady(elapsed:) on ConversationStartupState. Co-authored-by: Cursor <cursoragent@cursor.com>
These types only exist as associated values of ConversationState; keep them in the same file and drop the Models junk-drawer entries. Co-authored-by: Cursor <cursoragent@cursor.com>
Move audio, WebRTC, startup, and agent-state configuration into ConversationConfig.swift so session knobs live in one file. Leave AgentState as a separate runtime type. Co-authored-by: Cursor <cursoragent@cursor.com>
Colocate ConversationStartupState, ConversationStartupMetrics, and ConversationStartResult in one file. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
ChatWidget renders a floating launcher and a drawer with transcript, composer, mic toggle and a placeholder orb, driven by a trimmed ChatWidgetViewModel over ConversationClient. ChatWidgetController is the host control surface from the start. Examples/DemoApp exercises it end to end with a client tool. Co-authored-by: Cursor <cursoragent@cursor.com>
Port the procedural Metal orb from ElevenLabs components-swift (Apache-2.0), trimmed to the volume-driven view, and drive it from the SDK's mic and agent audio observers: a lock-protected RMS monitor per stream, sampled at 30 Hz so PCM-rate audio never re-renders SwiftUI. The mic button reuses the same level. Co-authored-by: Cursor <cursoragent@cursor.com>
Add WidgetConversationMode so a host can run the widget as voice-only, text-only, or both. Text-only conversations connect with audio disabled and start implicitly on the first message; voice-only drops the composer and keeps the orb above the transcript. The demo switches modes at runtime. Co-authored-by: Cursor <cursoragent@cursor.com>
renal128
force-pushed
the
widget-conversation-modes
branch
from
July 26, 2026 17:09
a4ace7f to
cc8e3d4
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit cc8e3d4. Configure here.
| guard !hasActiveConversation else { return } | ||
| let task = Task { | ||
| _ = try await client.startConversation(auth: authProvider(), config: conversationConfig) | ||
| _ = try await client.startConversation(auth: authProvider(), config: effectiveConversationConfig) |
There was a problem hiding this comment.
Mode read after auth await
Medium Severity
effectiveConversationConfig is taken inside the startup task around authProvider(), not when start/send begins. A mode change during that await can start a voice session from a text-only action (or the reverse), including an unexpected microphone permission prompt.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit cc8e3d4. Configure here.
renal128
force-pushed
the
widget-orb
branch
2 times, most recently
from
July 26, 2026 23:10
b8a6979 to
fcb4f18
Compare
jacksonh
approved these changes
Aug 3, 2026
renal128
force-pushed
the
widget-orb
branch
7 times, most recently
from
August 8, 2026 21:14
85cc784 to
4913cd2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
WidgetConversationMode(.voiceOnly,.textOnly,.voiceAndText) toChatWidgetConfigand makes the widget lay itself out around it..textOnlystarts the conversation withconversationOverrides.textOnly, so no microphone is touched and the call runs over WebSocket..voiceOnlydrops the composer and centers the call controls; the drawer leads with the large orb.Test plan
swift build,swift test,swiftformat . --strictxcodebuildofExamples/DemoAppfor the iOS simulatorStacked on #225.
Made with Cursor